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Abstract 

The purpose of this study was to identify whether the quality of afterschool 
educators varies across economic groups of students. This article describes a 
statewide study of the relationship of ratings on CLASS—a validated measure 
for teaching and learning interactions—and student economic status. In es¬ 
sence, what is the distribution of high-quality afterschool educators among an 
afterschool student population in Nebraska comprised of 3,791 elementary 
students? An analysis of variance showed that elementary students who were 
eligible for free meals experienced out of school time instruction from staff 
members who were rated of significantly lower quality in Emotional Support 
(ES) and Instructional Support (IS) on the CLASS. 
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Introduction 

Multiple studies have examined the relationship between socioeconomic sta¬ 
tus (SES) and academic achievement (Jensen, 2009). Children living in poverty 
experience multiple environmental risk factors that can and often do adverse¬ 
ly affect their academic skills (Lacour & Tissington, 2011). However, when 
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economically disadvantaged students regularly attend high-quality afterschool 
programs, they experience significant gains in achievement, work habits, and 
reductions in behavior problems (Vandell, 2013; Vandell, Reisner, & Pierce, 
2007). Meta-analyses conducted by Lauer and colleagues (2006) found small 
but significant positive effects of afterschool or summer school participation on 
reading and math achievement. Durlak and Weissberg (2007) found three areas 
of significant improvement for participants in afterschool programs: feelings 
and attitudes, behavioral adjustment, and school performance. 

This leads to the question: What is “high quality”? Metz, Goldsmith, and 
Arbreton (2008) proposed six dimensions of quality—focused, intention¬ 
al programming; continuous program improvement; exposure; supportive 
relationships; family engagement; and cultural competence. Of these six dimen¬ 
sions, Metz and colleagues asserted that two dimensions in particular—focused, 
intentional programming and continuous program improvement—were key 
in the formation of the others. Intentional programming is found in multiple 
reviews of quality in afterschool programs (Durlak & Weissberg, 2007; Little, 
Wimer, & Weiss, 2008). In an examination of the characteristics of quali¬ 
ty afterschool programs, Durlak and Weissberg (2007) reported that they are 
SAFE—sequenced, active, focused, and explicit. Converting that to effective 
teaching, then, one would say that effective afterschool educators implement 
activities that are sequenced, active, focused, and explicit. Indeed, research 
has consistently shown that one of the strongest assets of high-quality after- 
school programs are high-quality staff (McElvain, Judith, & Diedrich, 2005). 
For example, Little and colleagues (2008) found that high-quality programs 
were those that had appropriate supervision, structure, and well-prepared staff 
(Little et ah, 2008). Similarly, Pittman, Garza, Yohalem, and Artman (2008) 
found that supportive relationships and safety are primary to quality. 

In a multilevel analysis of the relationship between teacher quality and 
student achievement, Heck (2007) found that collective teacher quality was 
associated with differences in student outcomes such that students in schools 
with higher professional standards for teachers (i.e., certification, content 
knowledge, and performance criteria) had higher achievement levels in reading 
and math, compared to students in schools with lower professional standards. 
Notably, collective teacher quality made an even bigger difference in student 
achievement in school settings where targeted subgroups (i.e., low SES, under¬ 
represented racial/ethnic backgrounds, ELL participants) were more prevalent, 
underscoring the importance of higher quality teaching for at-risk students. 
Further support for these findings comes from Woodland (2008), who found 
that the area of adult-child relationships was first among nine core elements 
of effective afterschool programs for Black youth. In sum, research shows that 
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having higher quality teachers is associated with increased achievement in read¬ 
ing and math, as well as a reduction in the achievement gap between lower SES 
minorities and their White, higher SES counterparts. 

Despite the plethora of research showing the positive effects of high-quality 
afterschool programs on children and families, research indicates that children 
in low-income families continue to face limited access to high-quality after- 
school programs (Hipps & Ormsby, 2005, as cited in Norris-Holmes, 2008). 
For example, Peske and Haycock (2006) found that disadvantaged children 
were more likely to have less effective teachers. Specifically, the researchers 
found that low-income students were significantly more likely to have nov¬ 
ice teachers and/or teachers with less education in their content area. In a case 
study of 37 federally funded afterschool programs in a low-income urban dis¬ 
trict, Norris-Holmes (2008) found that only 30% of the afterschool instructors 
were identified as highly qualified teachers. Approximately 25% of the teach¬ 
ers were highly qualified in reading, 20.5% were highly qualified in math, and 
only 18% were highly qualified in both reading and math. 

Further, few states have effectively demonstrated that they have policies in 
place to prevent inequitable distribution of inexperienced teachers with stu¬ 
dents in poverty. Insufficient resources and high teacher turnover—conditions 
that are particularly prevalent in districts serving high concentrations of low- 
SES students—make it difficult for these districts to hire and retain highly 
qualified teachers (Goldhaber, 2002). Students of lower SES are almost twice 
as likely to have teachers with less than three years of teaching experience, 
compared to their higher SES counterparts (National Center for Education 
Statistics, 2002, as cited in Heck, 2007). Put simply, children from poverty 
backgrounds experience less than an equal share of high-quality teachers. 

This study set out to evaluate whether effective educators were distributed 
equally among students in poverty in Nebraska’s 21st Century Community 
Learning Center (CCLC) program. The U.S. Department of Education’s 21st 
CCLC program is authorized under Title IV, Part B of the Elementary and 
Secondary Education Act, amended by the No Child Left Behind Act of2001. 
This program provides federal funding to community learning centers to pro¬ 
vide academic, artistic, and cultural enrichment for students, particularly those 
at risk (such as students in poverty), in order to meet standards in core academ¬ 
ic areas such as reading, mathematics, and science. Funding grew immensely 
since its inception—from $40 million in 1998 to $1.09 billion in FY 2013. 
Funding is distributed to states, who administer funding to schools and com¬ 
munity partners through competitive grants (Afterschool Alliance, 2014). 
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Method 

Study Design 

This retrospective study was designed to determine if the quality of educators 
in afterschool programs was equitably distributed across economic groups. To 
address the distribution question, a measure of afterschool educator quality and 
a demographic variable to use as an indicator of economic status had to be iden¬ 
tified. Nebraska implements a comprehensive evaluation plan each year, and 
there were two possible data sources for quality and one for economic status. 

Program Model 

The Nebraska 21st CCLC program model requires that school-based and 
school-aligned afterschool and summer school programs focus on three goals: 
(1) student academic achievement, (2) student social/behavioral support, and 
(3) family and community engagement. Programs operate a minimum of 12 
hours per week after school and varying hours of programming in the summer. 

Measures 

The first potential quality data source was a locally developed program 
self-assessment rating tool (Nebraska Quality Out of School Time Program 
Self-Assessment Rating Tool, St. Clair, 2013). Because this tool was a self- 
assessment rating completed by members of the school-based afterschool 
program leadership team and because the tool was newly implemented, this 
data source was rejected as a valid and reliable means for calculating afterschool 
educator quality. 

The second potential quality data source was the Classroom Assessment and 
Scoring System or CLASS for kindergarten through third (and sometimes fifth) 
grade levels (Pianta, La Paro, & Hamre, 2008). The K—3 CLASS tool rates 
teaching/learning interactions and includes three domains: Emotional Sup¬ 
port, Organization, and Instructional Support. Nebraska’s 21st CCLC grant 
program broadly utilized the CLASS in 68 kindergarten through third (and 
sometimes fifth) grade out of school time programs. 

The second data source for quality—CLASS—was selected. The CLASS is 
a valid means of rating aspects of a teaching/learning environment, such as a 
classroom or afterschool learning group, that are of importance to impacting 
student achievement (Pianta et al., 2008). Further, CLASS is significantly and 
positively correlated with an internationally recognized and utilized environ¬ 
ment quality measure, the Environment Rating Scales (ECERS-R; Pianta et 
al., 2008). CLASS also demonstrates predictive validity related to students’ 
academic and social development (Howes et al., 2008). 
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As previously stated, the CLASS tool is divided into three domains—Emo¬ 
tional Support, Classroom Organization, and Instructional Support. These 
domains are further divided into 10 dimensions, each of which represents ele¬ 
ments of classroom quality that have been found to be influential in children’s 
learning. The domain of Emotional Support, which focuses on teachers’ abili¬ 
ties to support students’ social and emotional functioning in the classroom, 
consists of Positive Climate, Negative Climate, Teacher Sensitivity, and Regard 
for Student Perspectives. Classroom Organization—which consists of Behav¬ 
ior Management, Productivity, and Instructional Learning Formats—assesses 
a wide range of classroom processes related to to the organization and man¬ 
agement of students’ behavior, time, and attention in the classroom. Lastly, 
the domain of Instructional Support is comprised of Concept Development, 
Quality of Feedback, and Language Modeling and looks at teachers’ ability to 
implement their curriculum in a way that effectively supports cognitive and 
language development. Through CLASS, three ratings were gathered for each 
elementary 21st CCLC program—Emotional Support, Organization (of the 
learning environment or classroom organization), and Instructional Support. 
Ratings are established on a 1 to 7 scale with 6-7 representing high quality, 3-5 
representing mid quality, and 1-2 representing low quality. Ratings across these 
three domains are not combined into an aggregate or singular quality rating. 

To measure student economic status, the only available variable was student 
level free or reduced price meal status (FRPL). Students eligible for free or re¬ 
duced price meals would be considered to be in poverty, whereas students not 
eligible would be considered to not be in a low-SES household. About 75% of 
Nebraska’s 21st CCLC regular attenders were eligible for FRPL in the 2013- 
14 program year (St. Clair, 2014). In later analyses, students will be analyzed 
by being a part of one of three groups: free, reduced, or not free/reduced. One 
potential limitation to this data source includes the possibility that families 
may not have completed the necessary paperwork to qualify for FRPL but may 
still fall within a poverty category. 

Participants 

There were 3,791 elementary students included in the study (53% male, 
47% female). Many students were White or Caucasian (38%), followed by 
Hispanic/Latino (33%), Black or African American (19%), multiple races/ 
ethnicities (5%), Native American (4%), or Asian/Pacific Islander (1%). Ap¬ 
proximately 20% of the students were English language learners. Almost 
one-quarter of the students (23%) were verified for special education services. 
The most common categories of special education verification were specific 
learning disability (9% of the total sample) and speech language impairment 
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(8%), followed by behavioral disorder or other health impairment (both 2%), 
and developmental delay and mental impairment (1% each). Given that the 
21st CCLC program is designed to serve students most at risk for school fail¬ 
ure, it makes sense that nearly a quarter of the students would be verified with 
a disability of some type. 

The K-3 (which can also be used K-5) CLASS was completed on after¬ 
school educators from 68 elementary programs within 19 districts. Within 
these, two were urban districts, and the other districts were rural. Students 
from urban schools represented 57% of the student population. Elementary 
programs ranged from one educator working with four students to one educa¬ 
tor with 15 students. Across the 68 elementary programs, at least 340 educators 
were included in the study using digital recordings coded for CLASS. Educa¬ 
tors ranged from having a high school diploma or GED to having Master’s 
degrees. Experience ranged from none to 25 years. 

Procedure 

Students’ economic status (as measured by FRPL eligibility) was collected di¬ 
rectly from student information management systems across Nebraska through 
locally administered Microsoft Access databases. These databases are used in the 
data collection system for evaluating Nebraska’s 21st CCLC program. 

Each program submitted digital recordings of teaching and learning inter¬ 
actions representative of their program. Typically, programs submitted four 
15-minute increments of teaching and learning interactions. Only reliable 
CLASS raters were used. CLASS reliability is established through individuals 
completing a two-day training with trainers authorized by Teachstone, fol¬ 
lowed by online reliability tests which measure ratings within one score 80% 
or more of the time with no consistently “off” areas. Further, within the evalua¬ 
tion team, raters complete another reliability with an anchor (usually a CLASS 
trainer on the team) to within one score 90% or more of the time. 

Results 

Results from an analysis of variance of CLASS ratings by students’ FRPL 
status in school showed that elementary students who were eligible for free 
price meals experienced out of school time instruction from staff members who 
were rated as significantly lower in quality in Emotional Support (ES) and In¬ 
structional Support (IS) on the CLASS at the p < .05 [F{2, 3788) = 9A2,p < 
.001 for Emotional Support and F{ 2, 3687) = 5.22, p < .01 for Instructional 
Support (see Table 1). Post hoc comparisons using the Tukey HSD test indi¬ 
cated that the mean ratings in Emotional Support for students eligible for free 
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price meals (M = 5.58, SD = 0.66) and reduced price meals (M = 5.55, SD = 
0.71) were significantly lower than the mean rating in Emotional Support for 
students not eligible for free or reduced price meals [M = 5.69, SD = 0.68). 
Post hoc comparisons using the Tukey HSD test indicated that the mean rating 
in Instructional Support for students eligible for free price meals (M = 1.93, SD 
= 0.51) was significantly lower than the mean rating in Instructional Support 
for students not eligible for free or reduced price meals (M = 2.01, SD = 0.70). 
No significant differences were found between students eligible for reduced 
price meals and the other two groups in Organization (O). Taken together, 
these results suggest that students with the greatest risk for student academic 
achievement failure based on their economic background experienced signifi¬ 
cantly less emotional support and instructional support in their out of school 
time programs compared to their higher socioeconomic counterparts. 

Table 1. One-Way Analysis of Variance of CLASS Ratings by Students’ 

Meal Status in School 



Sum of 
Squares 

df 

Mean 

Square 

F 

Sig. 


Between 

Groups 

8.407 

2 

4.203 

9.418 

.000 

ES 

Within 

Groups 

1690.740 

3788 

.446 




Total 

1699.147 

3790 





Between 

Groups 

1.717 

2 

.858 

1.277 

.279 

O 

Within 

Groups 

2546.560 

3788 

.672 




Total 

2548.277 

3790 





Between 

Groups 

3.251 

2 

1.625 

5.223 

.005 

IS 

Within 

Groups 

1147.335 

3687 

.311 




Total 

1150.586 

3689 





Note: ES=Emotional Support, 0=Organization, IS=Instructional Support 


Discussion and Implications 

Elementary students who were eligible for free price meals experienced out 
of school time instruction from staff members who were rated of significantly 
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lower quality in Emotional Support (ES) and Instructional Support (IS) on the 
CLASS, though not significantly different in Organization (O, organization 
of the learning environment or classroom organization). This result is con¬ 
cerning, as relationships with teachers are linked to student engagement and 
achievement (Klem & Connell, 2004). Moreover, strong, positive teacher and 
student relationships are even more important for economically disadvantaged 
students (Roorda, Koomen, Spilt, & Oort, 2011). All students, but particu¬ 
larly economically disadvantaged students, need to be taught using higher level 
instructional activities with student-centered, experiential learning methods 
(Estes, 2004). Further, a system of education in the United States that prompts 
greater development of critical thinking skills in students is necessary to close 
the gap between U.S. students and those in Finland, Korea, Japan, Canada, 
Netherlands, Belgium, New Zealand, and Germany (Ripley, 2013). 

These findings highlight the need for a closer examination of staff selection 
policies across Nebraska’s afterschool programs, as well as an increased focus 
on ways to improve the quality of afterschool instructors, particularly for those 
teachers serving at-risk populations. One way would be to provide professional 
development (PD) to all Nebraska afterschool educators with additional target¬ 
ed focus on staff serving greater percentages of children and families in poverty. 
Research has consistently demonstrated the importance of PD for improving 
outcomes of early childhood programs; effective PD is associated with increases 
in teacher knowledge, student learning, and program quality (Christ & Wang, 
2013; Powell, Diamond, & Cockburn, 2013). 

Another path toward improvement might be to provide coaching to nov¬ 
ice educators and/or educators with lower CLASS ratings. Coaching is a form 
of PD that takes place directly in the classroom or learning environment and 
involves helping teachers acquire, improve, or refine specific, evidence-based 
intervention practices or teaching behaviors, as well as offering ongoing sup¬ 
port and individualized feedback (Hsieh, Hemmeter, McCollum, & Ostrosky, 
2009; Wasik & Hindman, 2011). Based on the Vygotskian concept of scaffold¬ 
ing, coaches work one-on-one with teachers to enhance their knowledge and 
understanding through a process of instruction, guided practice, and reflection 
(Wasik & Hindman, 2011). Coaching is frequently offered as part of a mul¬ 
ticomponent PD program that includes introductory workshops, an ongoing 
course, or web resources that offer information on evidence-based practices re¬ 
lated to the content of the PD (Powell et ah, 2013). 

Emerging research stipulates combining coaching with other forms of PD 
to effectively support teachers and improve outcomes for children. For exam¬ 
ple, Wasik and Hindman (2011) implemented the Exceptional Coaching for 
Early Language and Literacy (ExCELL) program in an effort to improve the 
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vocabulary and preliteracy skills of at-risk preschoolers. ExCELL was a PD 
program for teachers emphasizing strategies to build language and literacy 
skills with their students. After one academic year, it was found that teachers 
in the intervention group had created higher quality classroom environments 
as measured by the Early Language and Literacy Classroom Observation and 
CLASS and by videotapes of teachers’ classroom book readings. Specifical¬ 
ly, evidence from the CLASS measure indicated that intervention teachers 
modeled language more, provided important feedback to children on their 
language, and were more effective in fostering concept development in chil¬ 
dren (Wasik & Hindman, 2011). Their study also showed that children in 
the intervention group performed significantly better than comparison-group 
peers on measures of receptive vocabulary and phonological sensitivity. Thus, 
another avenue for future research would be to examine the relationship of 
educator quality and student achievement. Specifically, is there an association 
between higher CLASS ratings and state assessment results in reading, math¬ 
ematics, science, and writing? 

As the achievement gap between at-risk students and their more economi¬ 
cally advantaged counterparts widens, the need for high-quality programs that 
facilitate learning and enhance students’ academic skills is key. While after- 
school programs such as 21st CCLC are a promising approach to promoting 
student outcomes, this research demonstrates that differences exist in the qual¬ 
ity of these programs, with low-income students receiving significantly lower 
quality afterschool instruction than their higher income counterparts. Thus, 
the students who have been shown to benefit most from quality afterschool 
programs may be the ones least likely to receive it. While recognition of this 
imparity is a crucial first step, there is a strong need for more research regard¬ 
ing strategies for ensuring that all students, regardless of their socioeconomic 
status, have access to high-quality afterschool programs. 
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